Skip to content

verify: degrade on a rate-limited 429 instead of raising (0.4.1) - #5

Merged
imxinchengyou merged 1 commit into
mainfrom
verify-degrades-on-429
Oct 22, 2026
Merged

imxinchengyou merged 1 commit into
mainfrom
verify-degrades-on-429

Conversation

@imxinchengyou

Copy link
Copy Markdown
Owner

Step 1 of switching the hosted service's rate-limit response from 503 to the 429 that the API docs already promise.

The hosted API returns 429 in two different cases:

  • Edge rate limiter (nginx, 120 req/min per key): transient. It will send Retry-After.
  • Monthly verify quota used up (app/routers/verify.py): lasts until next month and needs a decision (top up or subscribe). No Retry-After.

Behaviour change for verify() / verify_batch()

  • A 429 with Retry-After now degrades like a 5xx: it logs a warning and returns degraded=True, with approved per fail_open (False by default), instead of raising.
  • A 429 without Retry-After (quota) still raises CacheVerifierError, as do other 4xx.
  • CacheVerifierError gets a retry_after attribute.
  • Control-plane calls are unchanged.

Today's 503 from the limiter already degrades, so this release changes nothing until the server switches.

Tests: rate-limited 429 degrades for single and batch calls; quota 429 (no header) still raises with retry_after is None; 401 still raises. 38 passed, 1 skipped; ruff clean.

Version 0.4.1: pyproject, __version__, CHANGELOG. Publish by creating a GitHub release v0.4.1, which triggers publish.yml.

A 429 that carries Retry-After is the edge rate limiter: transient load on
the caller's request path, which should fall through to the LLM like a 5xx
rather than throw. A 429 without Retry-After is the monthly verify quota
running out and still raises, since that needs a decision, not a silent
month of fallbacks; other 4xx still raise too. CacheVerifierError gains a
retry_after attribute.

Prepares for the hosted service switching its rate-limit response from 503
to the documented 429.
@imxinchengyou
imxinchengyou merged commit 4ac2693 into main Oct 22, 2026
7 checks passed
@imxinchengyou
imxinchengyou deleted the verify-degrades-on-429 branch October 22, 2026 09:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant